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ABSTRACT , . 

j A seven step strategy is presented for selecting a 

"good" test in an educational setting. The steps are as fellows: (1) 
wake the following preliminary decisions: (a) decide why you need to 
test, (b) decide what you intend to do with the results, (c) decide 
what practical decisions will be made based on the testing results, 
and (d) decide who will make those decisions and who will be affected 
by those decisions; (2) form a Test Selection and Review committee 
composed of decisionmakers and those who will be directly affected by 
the results of the testing; (3) search the fcroal literature for 
information about published and unpublished tests; C**) talk to test 
experts in other schools or colleges; (5) tentatively select one or 
more tests and pilot it; C6) evaluate the process and the products of 
the testing program; and (7) hold open the option to cycle back to 
any- previous stage as the need arises. (Author) 
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A SEVEN STEP STRATEGY FOR 
CHOOSING A "GOOD" TEST 



Section I. Introduction 

This short paper will present the text of an oral and written presenta- 
tion le by the author when serving as a chairperson of a panel 
durih , the State 1974 Testing Conference held at the Los Angeles, 
Convention Center on February 28, 1974. This conference on the 
use of tests and other measuring devises in education was attended 
by more than 373 educators. The main speakers at the conference 
were Dr. Peter Bentley, Professor of Psychology at the" University 
of California at Los Angeles and Dr. Williani Michael, Professor 
of Psychology and Education, University of Southern California. 
Dr. Behtley's address concerned "Reliability, Validity and Norming", 
and I^. Michael's address concerned " The Present and Future 
State of the Testing Art". Several special interest sessions were 
provided for attendees. These interest sessions included: (1) cog- 
nitive - language arts, (2) cognitive - mathematics and science, 
(3) cognitive - other, (4) affective, (5) career education, and (6) bi- 
lingual and/or disadvantaged. While :he present paper describes 
the presentation made by the writer who acted as chairperson of the 
panel which focused on the cognitive - mathematics, science interest 
section area, it is recognized that this si 'tegy is probably appro- 
priate for use in the selection and utilization of any test by any 
educational agency. 
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Section II. A Seven Step Stratej^y 
for Selecting; a "Good" Test 

This strategy includes the following seven steps : Step 1: Make the 
following preliminary decisions: (a) Decide why you need to test, 
(b) Decide what you intend to do with the results, (c) Decide what 
practical decisions will be made based on the testing results, and 
(d) Decide wbo will make those decisions, and who will be affected 
by those decisions; Step 2: Form a Test Selection and Review 
Committee composed of decision-makers and those who will be 
directly affected by the results of the testing; Step 3: Search the 
formal literature for infprmation about published and unpublished 
tests; Step 4: Talk to test experts in other schools or colleges; 
Step 5: Tentatively select one or more tests and pilot it; 
Step 6: Evaluate the process and the products of testing program; 
and Step 7: Hold open the option to cycle back to any previous stage 
as the need arises. 

Step 1: Make Preliminary Decisions. 

First of all one should ask himself: "Why is testing necessary?; 
What will be done with the results?; What practical decisions will 
be reached based on the results of testing?; Who will be making the 
decisions?; and finally, who will be most directly affected by those 
decisions?" 

Selecting the "best" test is similar to selecting the "best" carpenter's 
tool. Instead of asking "Which is best. Test A or Test B?" one 
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could ask a carpontor, "Which is "best", a hammer or a saw ?" 
Obviously the answer depends on what job is to be done by the 
carpenter's tool. In building a house a carpenter may view the half- 
framed structure and decide he wants to strengthen the studs by 
driving nails, so he grabs his hammer in order to do that particular 
job. While the notion of identifying the purpose of the activity first, 
appears to be an exercise in common sense, it is nevertheless an 
.mportant and necessary step which is often overlooked. Many times 
an educator finds himself holding a hammer and searching for some- 
thing ^ do with it. 

While answers to the questions of, "What decisions will be made based 
on the test results?, " and "Who will be most directly affected by the 
testing results?" may not be among the easiest answers to uncover, 
a little time spent on these will be well worth the time of the person 
charged with the reponsibility of test selection and use. 

Step 2: Establish a Test Selection and Review Committee. 
. isecondly, one should form a Test Selection and Review Committee. 
This committee should include representatives from two main groups: 
(1) those who will be making decisions based on the testing results, 
and (2) those who will be most directly affected by testing results. 
While one may initially be reluctant to attend "another meeting", one 
should recognize that the Test Selection and Review Committee can 
be very productive. This committee serves two important functions: 
(1) it forms a forum for communication to all affected segments, 
and (2) it can provide a "we" feeling among the participants. Much 
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attentitiii has boon focused on the use and abuse of tests, and the 
wise test selector will afford better communication, and broaden his 
base if he establishes and directs meetings of a Test Selection and 
Review Committee. A misinformed citizen may easily attack a 
single individual who has selected tests by himself alone, but that 
same misinformed citizen will probably calm down when he talks 
to a person who represents him and his group on the Test Selection 
and Review Committee. 

' Step 3:. Search the Formal Literature. 
The third step is to search the "literature". Several excellent books 
and booklets cover the testing instruments that are now being offered 
for sale to potential users. The Mental Measurements Yearbook 
is an important source book. The most recent edition was published 
in 1970. Another important book is entitled, Tests in Print. Both 
the Yearbook and Tests in Print are edited by Oscar Euros. The 
Dissemination Office of the Center for the Study of Evaluation at 
the University oi California at Los Angeles publishes a number of 
documents which also review tests. Among these UCLA documents 
are the following: (a) CSE-ECRC Preschool/Kindergarten Test 
Evaluations, (b) CSE Elementary School Test Evaluations, (c) CSE- 
HLP Test Evaluations: Test for Higher Order Cognitive Affective, 
and Interpersonal Skills. Soon to be published by the UCLA Center 
for the Study of Evaluation is a document which will deal with evalua- 
tion of tests which are used in secondary schools, in colleges and 
for adults. 
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In addition, virtually every test publisher will send basic information 
on catalogues about their test(s) to those who indicate an interest. 
Publishers usually offer for sale for a small fee ($2.00 to $5.00) 
a "specimen set" of materials relating to a test they sell. Moreover, 
test manuals are usually for sale for a small fee. Thus one could 
build a file "bank" of basic test information sheets, test catalogues, 
specimen sets, and test manuals. 

Step 4: Talk to the Experts. 

The fourth-s^ep tn this Test Selection Strategy is to talk to as many 

testing experts as possible. If one is located in a college or school 
district one can uncover several names of reputable test authorities 
in other colleges or school districts. It is in this step that one 
recognizes that the wheel or a portion of it may have already been 
I invented elsewhere. At the very least one will find that others are 
facing similar problems in certain ways which may or may not have 
some applicability to one's own local situation. 

Step 5: Do a Pilot Study. 

Ideally, once the array of test information has been established and 
reviewed by the Test Selection Committee, the committee should 
select one or more tests in order to do a pilot study. Ideally, the 
pilot study results would be reviewed by the committee before larger 
scale utilization of the test takes place. If the situation is not so 
ideal, and a decision on a test for large scale utilization is needed, 
the committee might find it advisable to use what is considered the 
"best" test on the required large scale and to use the "second best" 
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test on a smaller scale. This secontkkind of "piloting" would provide 
comparative information on both tests ^nd safeguard against putting 
all the "testing eggs" in one basket. 

Step 6: Evaluate. 

Step six involves evaluating both the process and product objectives 
of the testing program. Both high level administrators and those 
on the Test Selection and Review Committee should periodically take 
a hard look at the testing program including what it does well, what 
it does not do well, and what activities should be planned to improve 
it. Especially important here is the development of local indicators 
of the testing program's level of success. It will be extremely bene- 
ficial to establish local norms, and local indicators of the validity 
of tests and subtests for the total population that takes the test as 
well as for subgroups of that total population. Certain standard text- 
book information which is readily avaiiabjie may prove beneficial. 
(Best, 1970; Bloom, 1971; Cronback, I960; and Marascuilo, 1971; 
Worthen, 1973.) 

Step 7: Be "Dynamic". 

Finally, step seven involves holding open the option to cycle back 
to any previous step as the need arises. Local indicators of the 
success level of the program may point out areas which offer oppor- 
tunities for improvement. In addition, the needs which prompt the 
creation, implementation, and management of ,,the testing program 
probably will change over the years. This step is a reminder that 
the choosing and the using of tests in education are such important 
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processes that tUey must be open to considered intervention aimed 
at the improvement of services which are provided. 



Section III. Conclusion 



A Seven Step Strategy has been proposed for selecting and using a 
"good'* test in an educational setting. While the strategy was deve- 
loped for presentation at the Cognitive - Mathematics and Science 
Interest Session at the Southern California Testing Conference, it 
is recognized that this strategy may be appropriate to the selection 
and utilization! of any test or other measurement instrument by any 
educational agency. 
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